Improving End-to-End Speech Translation by Leveraging Auxiliary Speech and Text Data

We present a method for introducing text encoder into pre-trained end-to-end speech translation systems. It enhances the ability of adapting one modality (i.e., source-language speech) to another text). Thus, model can learn from both unlabeled and labeled data, especially when data is abundant. Beyond this, we denoising build robust that deal with normal noisy data. Our system sets new state-of-the-arts on MuST-C En-De, En-Fr, LibriSpeech En-Fr tasks.